The present invention relates to audio/video data processing, and more particularly to an audio/video processing engine that is optimized for the audio and video computations associated with processing video at either standard or high definition rates.
Previously video and audio data have been processed separately. In a production environment the audio and video are recorded separately, processed separately and then only united for transmission as a broadcast signal or for final display by an end user. Likewise test equipment for audio and video data developed independently. With the advent of digital video, digital audio may be embedded in the digital video as ancillary data for transmission. However before embedment and after extraction the audio and video are still processed separately. This separate processing presents some problems, such as xe2x80x9clip syncxe2x80x9d where the video and audio are noticeably out of sync when displayed to the end user. One attempt to solve this problem is represented by audio frame synchronizers, as disclosed in U.S. Pat. No. 4,313,135, where the video before and after processing is compared to determine a delay caused by the processing, which delay is used to likewise delay the audio to be in sync with the video. Another technique is disclosed in U.S. Pat. No. 4,963,967 where a coincidental marker is inserted into both the audio and video components so that the components may be synchronized at a receiver by realigning the coincidental markers. These techniques process the video and audio channels separately. Still another technique is disclosed in U.S. Pat. No. RE33,535 where a low resolution representation of the audio signal is inserted into the blanking interval of the video. However the blanking interval is stripped during digital processing, so the low resolution audio signal for synchronizing audio with video is lost.
However with the advent of watermarking technology, as disclosed in U.S. Pat. No. 4,969,041, an opportunity has been seen to use this technology to provide audio/video synchronization by inserting audio synchronization information in the form of a watermark into the active portion of the video U.S. Pat. Nos. 6,211,919 and 6,246,439). To perform this technology requires an audio/video processing engine that processes both audio and video simultaneously in real time.
Therefore what is desired is an audio/video processing engine that can process audio and video simultaneously in real time and is general purpose in nature so that by reprogramming any audio/video processing may be performed simultaneously.
Accordingly the present invention provides an audio/video processing engine that has a single programmable hardware processor having reconfigurable logic blocks for performing realtime processing of digital video and digital audio simultaneously. The digital video and digital audio are input to and output from an audio/video I/O processor that in turn is bi-directionally coupled to the programmable hardware processor. The programmable hardware processor processes the digital video and audio received from the A/V I/O processor with the assistance of an optional digital signal processor and a general purpose processor having a flash memory for adjunct processing, and outputs the results via the A/V I/O processor. Also appropriate auxiliary memories, such as a video buffer and a general purpose memory, may be coupled to the programmable hardware processor. The programmable hardware processor is coupled to access the flash memory so that constants don""t need to be downloaded into the programmable hardware processor. The general purpose processor is coupled to remote devices for control, testing and monitoring, and is used to remotely reload the flash memory according to the desired processing for the A/V processing engine.
The objects, advantages and other novel features of the present invention are apparent from the following detailed description when read in conjunction with the appended claims and attached drawing.